Acknowledgments

Is psychological science reproducible?

Databrary.org

  • Digital data library specialized for research video
  • Video/audio + participant/context metadata
  • Restricted access for research/educational use
  • Policy framework for sharing identifiable data

Video uniquely captures behavior

Video uniquely captures behavior

(Frank 2014)

Unequivocally & unambiguously demonstrates phenomena

Video depicts unreported methodological details (Adolph 2013)

Datavyu makes video coding reproducible

  • Raw research video must be coded
  • Datavyu.org = free, open source coding tool
  • Add codes, annotations time-locked to video segments
  • Ruby API for scripting reproducible workflows

Datavyu facilitates reproducible video coding

Databrary enhances reproducibility

  • "Active" curation of data as it is collected
  • Organize, share, standardized participant metadata
  • Sharing based on
    • user access level
    • participant permission

(Adolph 2013)

Standardized – reproducible – release levels

Databrary enhances reproducibility

  • Share raw research videos
  • Share video excerpts for teaching and learning
  • Share materials, coding spreadsheets, displays
  • Share links to papers, code repositories
  • Shared datasets get DOIs
  • Long-term preservation via partnership with NYU Libraries

Take homes

To boldly go where no science has gone before…

  • Reproducibility of numerical computation ≠
  • Reproducibility of behavioral measures, manipulations
  • All behavioral researchers should
    • share reproducible workflows of data, analyses AND
    • videos of behavioral measures, displays

Join us!

References

Adolph, Karen; 2013. “Infants Crawling and Walking over High and Low Bridges.” Databrary. doi:10.17910/B7MW2K.

Collaboration, Open Science. 2015. “Estimating the Reproducibility of Psychological.” Science 349 (6251): aac4716. doi:10.1126/science.aac4716.

DeLoache, Judy. 2014. “Scale Errors Offer Evidence for a Perception-Action Dissociation Early in Life.” Databrary. doi:10.17910/B7H019.

DeLoache, Judy S., David H. Uttal, and Karl S. Rosengren. 2004. “Scale Errors Offer Evidence for a Perception-Action Dissociation Early in Life.” Science 304 (5673): 1027–9. doi:10.1126/science.1093567.

Frank, Michael C.; 2014. “Representing Exact Number Visually Using Mental Abacus.” Databrary. doi:10.17910/B7PP4W.

Gilbert, Daniel T., Gary King, Stephen Pettigrew, and Timothy D. Wilson. 2016. “Comment on ‘Estimating the Reproducibility of Psychological Science’.” Science 351 (6277): 1037–7. doi:10.1126/science.aad7243.